Evaluating Language Translations: Experiments on Three Assessment Methods

Experiments were run to assess three ways of evaluating the quality of language translations: back translation, knowledge testing, and performance testing. Twelve professional English-to-Vietnamese translators processed approximately 10,000 words of technical material (i.e., a helicopter maintenance manual). Subjects took knowledge tests or performed a difficult maintenance task using translated materials. Vietnamese Air Force technicians and U.S. Army technicians served as primary subjects and controls, respectively. The analysis of back translations showed the frequency and types of translation errors that occurred. Knowledge test scores satisfactorily discriminated different quality levels of translations. The performance tests demonstrated (a) the impact of translation quality on performance, (b) the value of working in one's native language (vs. having to learn English), and (c) the importance of providing high-fidelity translations where a complex task is to be done.